Observing the clouds : a survey and taxonomy of cloud monitoring

This research was supported by a Royal Society Industry Fellowship and an Amazon Web Services (AWS) grant. Date of Acceptance: 10/12/2014Monitoring is an important aspect of designing and maintaining large-scale systems. Cloud computing presents a unique set of challenges to monitoring including: on-demand infrastructure, unprecedented scalability, rapid elasticity and performance uncertainty. There are a wide range of monitoring tools originating from cluster and high-performance computing, grid computing and enterprise computing, as well as a series of newer bespoke tools, which have been designed exclusively for cloud monitoring. These tools express a number of common elements and designs, which address the demands of cloud monitoring to various degrees. This paper performs an exhaustive survey of contemporary monitoring tools from which we derive a taxonomy, which examines how effectively existing tools and designs meet the challenges of cloud monitoring. We conclude by examining the socio-technical aspects of monitoring, and investigate the engineering challenges and practices behind implementing monitoring strategies for cloud computing.Publisher PDFPeer reviewe

Improving resource efficiency of container-instance clusters on clouds

Cloud computing providers such as Amazon and Google have recently begun offering container-instances, which provide an efficient route to application deployment within a lightweight, isolated and well-defined execution environment.Cloud providers currently offer Container Service Platforms (CSPs), which orchestrate containerised applications.Existing CSP frameworks do not offer any form of intelligent resource scheduling: applications are usually scheduled individually, rather than taking a holistic view of all registered applications and available resources in the cloud. This can result in increased execution times for applications, resource wastage through underutilised container-instances, and a reduction in the number of applications that can be deployed, given the available resources.The research presented in this paper aims to extend existing systems by adding a cloud-based Container Management Service (CMS) framework that offers increased deployment density, scalability and resource efficiency. CMS provides additional functionalities for orchestrating containerised applications by joint optimisation of sets of containerised applications, and resource pool in multiple (geographical distributed) cloud regions.We evaluated CMS on a cloud-based CSP i.e., Amazon EC2 Container Management Service (ECS) and conducted extensive experiments using sets of CPU and Memory intensive containerised applications against the direct deployment strategy of Amazon ECS. The results show that CMS achieves up to 25% higher cluster utilisation, and up to 70% reduction in execution times.Postprin

Resource efficiency in container-instance clusters

This research is supported by the Amazon Web Services (AWS) Education Research Grant.Cloud computing providers have recently begun offering container instances, which provide an efficient route to application deployment within a lightweight, isolated and well-defined execution environment. Cloud providers currently offer Container Service Platforms (CSPs), which support the flexible orchestration of containerised applications. Existing CSP frameworks do not offer any form of intelligent resource scheduling: applications are usually scheduled individually,rather than taking a holistic view of all registered applications and available resources in the cloud. This can result in increased execution times for applications, resource wastage through underutilized container-instances, and a reduction in the number of applications that can be deployed, given the available resources. This paper presents a cloud-based Container Management Service(CMS) framework, which offers increased deployment density, scalability and resource efficiency for containerised applications. CMS extends the state-of-the-art by providing additional functionalities for orchestrating containerised applications by joint optimisation of sets of containerised applications and resource pool on the cloud. We evaluate CMS on a cloud-based CSP i.e., Amazon EC2 Container Management Service (ECS) and conducted extensive experiments using sets of CPU and Memory intensive containerised applications against the direct deployment strategy of Amazon ECS. The results show that CMS achieves up to 25% higher cluster utilisation and up to 2.5 times faster execution times.Postprin

FIFE: an Infrastructure-as-code based Framework for Evaluating VM instances from multiple clouds

Funding: ABC project (Adaptive Brokerage for the Cloud) funded by EPSRC EP/R010528/1.To choose an optimal VM, Cloud users often need to step a process of evaluating the performance of VMs by benchmarking or running a black-box search technique such as Bayesian optimisation. To facilitate the process, we develop a generic and highly configurable Framework with Infrastructure-as-Code (IaC) support For VM Evaluation (FIFE). FIFE abstract the process as a searcher, selector, deployer and interpreter. It allows users to specify the target VM sets and evaluation objectives with JSON to automate the process. We demonstrate the use of the framework by setting up of a Bayesian optimization VM searching system. We evaluate the system with various experimental setups, i.e. different combinations of cloud provider numbers and parallel search. The results show that the search efficiency remains the same for the case when the search space is consist of VM from multiple cloud providers, and the parallel search can significantly reduce search time when the number of parallelisation is set properly.Postprin

Benchmarking and performance modelling of MapReduce communication pattern

Funding: UK EPSRC EP/R010528/1 and IsDBUnderstanding and predicting the performance of big data applications running in the cloud or on-premises could help minimise the overall cost of operations and provide opportunities in efforts to identify performance bottlenecks. The complexity of the low-level internals of big data frameworks and the ubiquity of application and workload configuration parameters makes it challenging and expensive to come up with comprehensive performance modelling solutions. In this paper, instead of focusing on a wide range of configurable parameters, we studied the low-level internals of the MapReduce communication pattern and used a minimal set of performance drivers to develop a set of phase level parametric models for approximating the execution time of a given application on a given cluster. Models can be used to infer the performance of unseen applications and approximate their performance when an arbitrary dataset is used as input. Our approach is validated by running empirical experiments in two setups. On average, the error rate in both setups is ±10% from the measured values.Postprin

Algorithms for optimising heterogeneous Cloud virtual machine clusters

This research was supported by an Amazon Web Services Education Research grant.It is challenging to execute an application in a heterogeneous cloud cluster, which consists of multiple types of virtual machines with different performance capabilities and prices. This paper aims to mitigate this challenge by proposing a scheduling mechanism to optimise the execution of Bag-of-Task jobs on a heterogeneous cloud cluster. The proposed scheduler considers two approaches to select suitable cloud resources for executing a user application while satisfying pre-defined Service Level Objectives (SLOs) both in terms of execution deadline and minimising monetary cost. Additionally, a mechanism for dynamic re-assignment of jobs during execution is presented to resolve potential violation of SLOs. Experimental studies are performed both in simulation and on a public cloud using real-world applications. The results highlight that our scheduling approaches result in cost saving of up to 31% in comparison to naive approaches that only employ a single type of virtual machine in a homogeneous cluster. Dynamic reassignment completely prevents deadline violation in the best-case and reduces deadline violations by 95% in the worst-case scenario.Postprin

Modelling VM latent characteristics and predicting application performance using semi-supervised non-negative matrix factorization

Funding: This work is a part of the ABC (Adaptive Brokerage for the Cloudproject) funded by EPSRC EP/R010528/1.Selecting a suitable VM instance type for an application can be difficult task because of the number of options and the variety of application requirements. Recent research takes a data-driven approach to model VM performance, but this requires carefully choosing a small set of relevant benchmarks as input. We propose a semi-supervised matrix-factorization-based latent variable approach to predict the performance of an unknown new application. This method allows to take a large set of benchmarks as input for VM performance modelling, and it uses the model and the performance measure of the new application on some of the target VMs to predict the performance on the rest of all VMs. We ran experiments with 373 micro-benchmarks from stress-ng and 37 AWS EC2 VMs to predict the scores of Geekbench accurately. Our initial results showed that the RMSE and STD of the predicted scores are 6.7 and 4.5 when sampling Geekbench on 5 VMs, and 10.0 and 2.8 when sampling 10.Postprin

Degradasi pada Beton HVSA-SCC akibat terpapar asam sulfat pengaruh konsentrasi asam sulfat dan kandungan fly ash dalam beton

ABSTRAK Nur Attika, 2016, Degradasi Beton HVFA-SCC Akibat Terpapar Asam Sulfat: Pengaruh Konsentrasi Asam Sulfat dan Kandungan Fly Ash Dalam Beton. Skripsi Program Studi Teknik Sipil Fakultas Teknik, Universitas Sebelas Maret Surakarta. Struktur beton diharapkan memiliki keawetan dan ketahanan yang baik selama waktu yang direncanakan. Kurangnya ketahanan disebabkan oleh banyak faktor, salah satunya berupa serangan zat kimia seperti asam sulfat. Serangan asam sulfat biasanya terjadi pada bangunan laut yang airnya mengandung klorida dan sulfat seperti pada pelabuhan, bangunan tepi pantai, pengeboran lepas, bangunan pengolahan limbah, dan industri air tanah. Asam sulfat merupakan zat kimia yang memiliki agresifitas yang cukup tinggi yang dapat menyebabkan degradasi pada beton. Bahan tambah fly ash dapat mengurangi dampak reaksi pasta semen pada beton dengan asam sulfat, meningkatkan kepadatan, dan workability. Penggunaan self compacting concrete juga akan menghasilkan beton dengan workability yang baik dan mengurangi penggunaan air. Kadar penggunaan fly ash pada beton hingga lebih dari 50% disebut dengan high volume fly ash- self compacting concrete. Kadar fly ash sebagai pengganti sebagian semen dalam beton yang cukup tinggi mampu memperkecil ruang antar agregat sehingga mengurangi reaksi beton dengan asam sulfat yang akan melarutkannya. Untuk observasi pengujian kuat tekan digunakan benda uji silinder diameter 15 cm dan tinggi 30 cm sebanyak 42 buah benda uji. Observasi perubahan fisik dengan foto digital dan observasi perubahan dimensi dan berat menggunakan benda uji silinder diameter 7,5 cm dan tinggi 15 cm sebanyak 27 buah. Pembuatan benda uji dilakukan variasi penambahan kadar fly ash, yang digunakan adalah 50%, 55%, 60%, 65%, dan 70% total berat benda uji, yang kemudian akan dipaparkan asam sulfat dengankonsentrasi 0%-4%. Dari hasil penelitian dapat diketahui bahwa konsentrasi asam sulfat memepengaruhi degradasi pada beton. Semakin tinggi konsentrasi asam sulfat maka semakin tinggi degradasi yang terjadi pada beton. Kadar fly ash juga mempengaruhi degradasi pada beton yaitu semakin tinggi kadar fly ash maka semakin kecil degradasi yang terjadi pada beton. Kata kunci : Degradasi, fly ash, HVFA-SCC, asam sulfa

Container-based Cloud Virtual Machine benchmarking

This research was pursued under the EPSRC grant, EP/K015745/1, ‘Working Together: Constraint Programming and Cloud Computing,’ an Erasmus Mundus Master’s scholarship and an Amazon Web Services Education Research grant.With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed. In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-definedportion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first modeas a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs.Postprin

Simplified cloud instance selection

Cloud computing delivers computational services to anyone over the internet. The cloud providers offer these services through a simplified billing model where customers can rent services based on the types of computing power they require. However, given the vast choice, it is complicated for a user to select the optimal instance types for a given workload or application. In this paper, we propose a user-friendly cloud instance recommendation system, which given a set of weighted coefficients representing the relevance of CPU, memory, storage and network along with a price, will recommend the best performing instances. The system only requires provider specified data about instance types and doesn’t require costly cloud benchmarking. We evaluate our approach on Microsoft Azure across a number of different common workload types.Postprin